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for Cognitive Learning is to help learners develpp as rapidly 
and effectively as possible their potential as human beings 
and as contributing members of society.^ The R&D Center is 
striving to fulfill this goal by 

• conductin^research to discover more about 
/ how children learn 

• developing improved instructional strategies, 
processes and materials for school administrators, 
teachers, and children, and 

• offering assistance to educators and citizens 
which will help transfer the outcomes of research 

and development" into practice ^ 



PROGRAM 

The activities of the Wisconsin R&D Center are organized 
around one unifying theme. Individually Guide3 Education. 
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ABSTRACT 



A continuation of the field test of Developing Mathematical Processes was 
conducted in eight schools. Four were multiunit schools located iny settings 
ranging from small town to large city; the remaining schools were conventionally 
organized and located in large urban areas. The purpose of the field test was 
(a) tor determine the effectiveness of the instructional program in terms of 
student achievement, and (b) to document the usability of the program. The 
second year of the field test focused on the third level of the program. 

The results of the field test indicated a mastery level of^ approximately 
81 percent on the specific objec*tives of the program. Overall performance 
was retarded by poor results m one topic. On the latter portions of th^ 
program maintenance of the mastery level reached a level of 83 percent. The 
data on standardized test achievement did not conclusively favor either DMP or 
its conventional counterparts. There was some evidence of lower achievement' 
for DMP -students in the nonurban multiunit schools. When the opportunity for 
a student to learn tested material was considered in the analysis, D^^ students 
compared favorably to non-DMP students. ^Finally, in regard to the usability of 
the program, the field test established that teachers will expend the effort to 
attend frequent inservice meetings, to prepare an appreciable amount of instruc- 
tional materials, and to plan for the several instructional modes in the program. 
In addition, it was determined that students will enthusiastically participate 
in the learning activities. On the basis of the data gathered in the field 
test. Developing Mathematical Processes was demonstrated to be a viable program 
for -elementary school children at the s^condf-grade level. 
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INTRODUCTION 



i 

This report describes the second year of a smal^l-scale field t$s1*of 
Developin<g Mathematical Processes (DMP) . The report on the first year of 
the field test (Hubbard, 1972) deals with implementation of the first two 
levels of the program with kindergartners and first graders. This report 

, deals with the use of the third level with second graders in the same schools 
Because the two years of the field test are similar, there is a body of de- 
scriptive material that pertains to both reports — it is included in both of 
them so that they may be read independently. 

The field test described in this report is designated a small-scale 
field test. It is the fourth phase of a five-phase program development 
sequence utilized by the DMP project. The phases are: definition, analysis, 
pilot, small-scale field test, and large-scale field test. The latter three 
phases involve the use 6f the program with students and teachers for the 
purpose of testing the program's effectiveness and usability. The two field 
test phases frequently involve the same version of the program, but there are 
Mifferences in the number and proximity of field test schools and in the 

♦ degree of mediation in the program by the R&D Center staff. Reports for 
each of the phases are available from the R&D Center. 

The version of the program used by the small-scale field test schools 
is designated the developmental version. It is a revision of the pilot 
version and antedates the commercial .version. Although there is a basic 
likeness among the three versions, there are differences in fortnat and in 
packaging as well as revisions based on the evidence gathered in each phase. 

OVERVIEW ^ 

Deve loping Mathematical Processes is a research-based instructional 
program for K-6 elementary school children that is being developed at the 
Wisconsin Research and Development Center for Cognitive Learning of the 
University of Wisconsin. A complete description of the program is not 
practical for this report, but several important characteristics are noted. 
For a more complete description see the Resource Manual, Topics 1-40, (1974) . 

DMP utilizes an activity context for learning mathematics — a carefully 
sequenced program of activities that provides instruction for a set of well- 
defined behavioral objectives. A variety of activities suited to the age 
and interests of the students is presented to involve them in "doing" 
mathematics. 

DMP approaches mathematics through measurement. Beginning with the 
object^ in their own world, the students examine the attributes of these 
objects and explore the relationships between them through the use of various 
processes using pictorial and symbolic representations of the objects and 
their interrelationships as the next step toward abstract mathematics. This 
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is follOHed by modeling mathematical sentences with real objects- Thus the 
connection between abstract mathematics ^ and the^ real world is continually 
emphasized in the measurement approach. This connection is further strength- 
ened by th^ problem-solving situations that arfe used in the DMP materials. 

DMP^integrates the mathematical areas of arithmetic, geometry, prob- 
ability, and statistics. This integration is a natural outgrowth of examin- 
ing the relationships between objects and generating numerical data from 
.real-world problems. DMP therefore includes the iniiuitive and informal 
geometry of size, shape, and relationships among two- and three-dimensional 
objects, and the elementary notions of probability and statistics that lead 
to the organization and analysis of data. 

DMP provides an instructional management system to^accommodate individ- 
ual differences in students. In addition to the variety of activities to 
siyit students* interests and learning ^yles, there are assessment ^materials 
that include observational procedures for using classroom activities to 
evaluate student progress and criterion-referenced tests for placement and 
evaluation. There are also sequencing suggestions that enable a teacher to 
select options for a student's progress through the program. This instruc- 
tional management system is consistent with the more comprehensive system 
of educational programs called Individually Guided Education (KlaMsmeierf 
Quilling, Sorfenson, Way, & Glasrud, 1971). The IGE system is designed for 
the individual student in sctch a way that planned variation^ are made in 
what the student learns, the rate at which he learns it, and the way he 
learns it. 

DMP includes ,an inservice program to assist teachers in the implementa- 
tion of the program. Since DMP cannot be successfull>)^implemented in schools 
without careful consideration by every teacher of the distinctions*.that sepa- 
rate DMP from other mathematics programs, a systematic ""iterative inservice 
program has been developed and field tested. The inservice consists of a 
series of meetings using audiovisual jriaterials and guides which describe 
th6 most effective teacher/learning environments and provide the background 
information necessary to implement the program. For more detailed informa- 
tion related to the implementation of DMP, see WorJcing Paper No. 74 (Romberg, 
McLeod,^ & Montgomery, 1971). The inservice was field tested as* a part of the 
first yecur of the small-scale field test. It is reported both in the field 
test report for the first year and in Technical Report No. 245 (McLeod) . 



GOALS AND OBJECTIVES OF DMP 

Terminal Goals ' ^ 

The primary goal of DMP is that, upon completion of the program, students 
will be able to translate problems from the everyday world into mathematics, 
solve the problems mathematically, and translate the results back into the 
everyday world. This goal is based on the belief tha/t the practical value 
of mathematics is in its application to reality. Students who are mathemat- 
ically proficient are more than computers? they are problem solv5brs. 

A second terminal goal is that, upon completion of the program, students 
will be able to examine mathematics, identify the structural properties and 
relationships in mathematics, and logically validate mathematical assertions. 
This goal can be reached only after students have had an opportunity to develop 
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considerable knowledge of mathematics^ Only then can triey begin to examine 
the characteristics of different systems and the properties of those systems. 

Specific Objectives ^ 

Each topic of DMP contains a set of specific behavioral objectives. In 
the assessment of these objectives, conditions are established for three 
levels of performance: Mastery (M) , Making Progress (P) , and ^fee4s Consider^ 
able Help (N) . Upon ^completion of instruction in a given topic a student is 
expected to perform at eithe*r of the first two levels of mastery (M or P) . 
For any given group completing a topic , 95 percent of the ratings of the 
students* performance are expected to be at the first two levels of mastery. 

Tl-iis" criterion represents a change from the criterion used in the first 
year of the field test when collectiqns of objectives were considered jointly 
as a way of estimating the students* overall performance for a complete .year ' s 
work. This proved to be difficult to assess accurately. More importantly., 
the specification of a year's work was inconsistent with the assumptions of 
the IGE system related to the continuous progress of learning. Thus the 
criterion was changed to focus more specifically on the immediate outcomes 
of instruction. 

The change in criterion does not mean that long-range evaluation or even 
intermediate-range evaluation is not of concern to the R&D Center staff. 
Specification of the terminal goals provides an appropriate context for 
assessing the long-range effects of the program. To provide for assessment 
of intermediate-range effects, accountability tests are being developed. In 
addition, teachers have been alerted to watch for students whose performance 
over the course of several topics does not show tendency toward mastery (M) . 

The change in criterion reflects two things; (1) that the primary con- 
cern at this stage of, development is on the immediate effect of instruction, 
and (2) that the field test is not designed t6 .assess the longitudinal per- 
formance of students in an IGE curriculum. 



SEGMENTS OF THE DMP PROGRAM ' 

The following three segments were developed for the field te^t version 
of the DMP program: 

1. Instructional materials > 

• The Teacher's Guide is. the primary source of information and guidance 
fpr implementing the program. It is a detail^^d exposition of objectives, 
activities, instructional recommendations, and supplemental suggestions. 
^The teacher uses the guide extensively when implementing the program. 

•The materials kit contains the item's of special ^ncFnuf acture that are 
essential to carrying out the instructional activities. 

•The printed materials package contains activity cards, pupil performance 
records, and other nonconsumable student materia^^s. 

/ 

•Pupil workbooks contain printed sheets that each student can use, at 
the discretion of the teacher, for practice and exploration. 
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Assessment materials 
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• An assessment manual provides assistance for usihg^the 
instruments. 



^ssment 



•Pupil *test booklets contain placemeht inventories for making judgment! 
about the ihitial grouping of students for instruction and topic inven- 
tories for assessing the students' mastery of the objectives. 

•Pupil performance records (found in the printed materials package) 
provide teachers with a number of opt:^ons for recording student , * 
achievement. ^ 

•Teacher Observation Procedures (contained in the Teacher's Guide) are 
a set of (JetaiS^d instructions for assessing the students' at^tainment 
of objectives as they participate in classroom activities. 



Inservice . materials 

• An Activity^pproach to Math describes the teaching stylets of the/\ 
teacher and the learning styles of the student that th^ developers 
have found' to be the most effective for implementation^o'f the pro<^ram. 

• Assessment and Managing Instrmction outlines a strategy that teachers 

^ can employ using DMP materials to adapt their instruction to the individ- 
ual achievement levels of the students. 

• DMP Sampler" providers an^overview pf - the program through the inclusion 
of sample pages from each of the instructional and assessment segments 
ol the program. » / ' 



REQWSITES FOR EFFECTIVE IMPLEMENTATION 

The R-& D Center 'fe.taff considers the following conditions to be requisite 
for effective implementation of DMP by field td^t schools: 

1. Attendance of the teaching staff (/ttendanc^'of princ4.pals is optional* but 
desirable) at an mservice meeting* sponsored by the developer and held 

^ ""V^ior to inlplementation of the program 

2. Instruction in 'DJ^ for at least hours ppr week for students in their 
third year of school ^ 

3. Participation' of the teaching staff in conferences and consultations with 
' the inservice 'coordinator under mutually convenient arrangements 

4. ^Demonstration of a teaching style that includes the following components: 
a. providing materials for activities 

■» ' 
i>. opening activities by posing problems or demonstrating an activity 
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c. grouping students for effective instruction, according to recommendations 
in the Jeache^'s Guide 

d. moving from groyp to group. and serving as* a. resource to |>articular needs 

t* ^ • 

e. conducting discussions -for* summarizing and extending ideas of an 
. activity * ' 

Attention to the assessment procedures and classroom management guidelines, 
^'^-%mich include: 

a. use of an observational/pupil-performance-record procedure to assess . 
students' progress 

b. use of topic inventories to substantiate teacher judgment and, in con- , 
junction with the plaeemerit inventories, to determine the mastery- 
levels of students 

c. selection of activities for instiniction in regard to the appropriateness 
to a given objective and the existing dis'tribution of student performance 
levels 

d. formation of instructional groups based upon judgments of each student's 
mastery level* and learning style ^ 

6. Attendance of the teaching staff (attendance of principals is optional but 
desirable) at a midyear inservice meeting sponsored by the developer 

OBJECTIVES of' THE FIELD TEST 

The principal objective of a field test is to determine whether thfe program 
meets, its own objectives when installed and implemented according to plan.* In 
addition a field test may have objectives which extend the design and instru- 
mentation beyond that required to evaluate the program objectives. 

For the DMP field test three additional objectives were, specified. Two 
o% these involved student performance on standardized tests and the maintenance 
of mastery levels of specific objectives. The. third objective focused on 
implemen-tation considerations—those aspects of any program of concern to 
school personnel when adoption of the program is contemplated — cost, staff 
responsibilities, time allocation, and usability of materials. ^\ 

The objectives of the field test were as follows: 

1. ' To 'determine whether the objectives of the program (described above) are met 

*, * 

2. To compare the achievement of students using DMP to students not usiftg DMP 
as measured by norm-referenced, standardized measures and Dy criterion- 
referenced, DMP achievement measures 

3. To determine the maintenance of the students' achievement on DMP objectives 
' ' previously mastered, •, ' . . . • 
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" '4.* To' .(iocxiRient fours aspects of DMP' implementation 
cost ^> 
st^ff •responsibilities 
• <7. time allocation • 
d/, *if§ability of materials 



SUBJECTS, ■ •• ^ . A • , 

"V , ' ' * 

The eight schools that participated in the first year of the field test 
also participated in th^ second year* Fotpr, of the schools we^j:^ located in 
cjentrarl city areas' and were organized conventionally into self-contained class- 
rooms*. The other four §chQols utilized the multiunit pattern of school organiza 
tion. One of . the multipnit 5.chools_was in an urban but noncentral-city area, 
while the remaining thi^ee schools -j^ere in settings that rarlged from small town 
to medium^si-zed city. . Central city and multiunit schools were chosen because 
of the Center's particular int^rept in deterrnining the program* s 'effectiveness 
in these settings. ^_ ^ 

students who used.DMP during the fi^st year of the field test* continued 
to use'^t during the second year. Of particular interest were those students 
in their third year of school (Grade^.2)* In the four central city schools, 
students in the third grade also usecJ'DMP bvft at the same level as the other 
participating students. In the three Chicago schools the third grade students 
were new to DMP, but the Milwaukee school had used DMP with its second graders 
during 1971-72, As shown in Table 1, approximately 1,000 children and 33 
teachers were included in the field test. 

The field test teachers in almost all cases were new to DMP. In spring 
1972 most of them attended meetings of approximately two-hours duration at 
which DMP and the fiel(3 test were described. A memorandum of agreement 
(Appendix A) was discussed with the teachers and subsequently signed by the 
appropriate administrator. 

INSTRUMENTATION , ' ' 

Specific instruments are associated with each objective of the field test. 
Objective 1^ 

The terminal goals of the program will be assessed by instruments to be 
developed for administration in later stages 'of the field test. These instru- 
ments will be designed to structure situations in which the student can demon- 
strate his ability to perform tasks that indicate attainment of the terminal 
goals. 

Specific program objectives were assessed by using the topic inventories, 
which are criterion-referenced instruments. Some specific objectives, however T 
could be assessed only by teacher obs.ervations ; these observations were recorded 
in the pupil performance records butTnot used for this objective. 



7 



17 



w 
u 

o 

s: 

- Eh 
<D 

w 

rH 

o 



w 

o 
Eh 

W 
0) 
05 



C 
0 
•H 
-P 
05 
N 
'H 

u 
o 



c 

0 
•H 

•H 

V4 

U 
0) 

(D 
Q 



CO. 



in m 



rH 


rH 


rH 


rH 


05 


05 


05 


05 


C 


C 


C 


C 


0 


0 


0 


0 


•H 


•H 


■H 


•H 






-P 


-P 


C 


C 


C 


C 








0 


> 


> 


> 


> 


c 




C 


c 


0 


0 • 


0 


0 


u 


u 


u 


u 



05 
-P 
0 
-P 



>1 
-p 

•H 

U 



05 
U 
-P 

c 

q; 
u 



-p 

•H 



05 
U 

c 

q; 
u 



00 I I 1 



D D D 

s s s 



mm m^rH CN ^ <n 



D 







rH 


>1 










05 


-p 










U 


'H 










^ 


U 










C 










>1 


OJ 


TJ 






-p 


-p 


u 


QJ 






•H 


•H 


c 


tg 


>i 


c 


U 


U 


0 


•H 


-P 








c 


W 


•H 


0 


rH 


rH 




1 


U 


-p 


05 


05 












Vh 


c 




rH 


rH 


-P 


-P 


05 


•H 


rH 


rH 


C 


c 








OJ 


0 




u 


o; 




B 


0 


U 


0 


S 















q; 


















a; 




05 


05 


















-P 












0- 






Vh 












05 




C 


05 




-P 












q; 


04 




•H 








rH 


q; 


o 






O 








•H 


q; 


u 






0 


0 


0 


IS 










TJ 


0^ 


CP 


CP 








c 




C 


05 


05 


05 




05 




0 




03 


O 


O 


u 






c 


0) 






•H 


•H 






rH 


H 


? 




rH 






s: 


q; 


•H 


4J 






0 


o 


o 


u 


u 


2 






q; 


0 






-p 




Ma 




rH 


x: 












QJ 


rH 


u 




c 






>i 




U 


•H 


w 


Vh 


05 


c 


x: 


U 


c 


c 


> 




05 


B 


•H 


-p 


0 


05 


a; 






rH 


X 


rH 




-p 




u 


o; 






U 




:3 


u 






rH 




•H 


05 


05 


0 


•H 






05 








a: 


Ct4 


> 


XL. 







CD 



00 



n ID 

.rH <N 



rH <N 



05 
-P 
0 
-P 
XI 



05 
-P 




Objective 2 

— nr^ ' 

Q 

The achievements of students who have used DMP and those who have not 
used DMP yere assessed by standardized measures, and th^se results were com- 
pared. The arithmetic sections of the California Achievement Test (Forms 
1-A and IB) and the Cooperative Primary Test (Forms 23A and 23B) were selected 

most closely reflecting the content of DMP for Grade 2 students. The 
appropriate forms of the same instruments will also be used for Grade 3 
students. In subsequent years of the field test the mathematical subtests ^ 
of 't^he Iowa Test of Basic Skills will be used at the upper grade levels. 
Accoxantability instruments are being developed by the project staff. . They 
will be used to assess, the stxxJents' overall achievement inr such areas as 
computation, problem solving, and understanding of mathematical concepts. 
Norms from several populations will be established for the instruments, but 
they will emphasize the content, notation, and sequence of development used, 
in DMP. These instrument^ were not available .for this field test. 

Objective 3^ 



A«teKc 



period of time had elapsed following instruction on an objective, 

the topic inventories were used to assess the mastejy level of students on that 

specific program objective. ' 1 

r 

Objective 4_ 

Data concerning the implementation of the program were gathered from, 
questionnaires and interview schedules developed by the R&D Center evalviA- 
tion staff for the purposes deemed important at the time of a monitoring: visit. 

/ 

TESTING -AND MONITORING SEQUENCE ^ 



' In a small-scale field test there is need for comprehensive information 
regarding the field test objectives. This, need is. offset, however, by the 
desire to minimize disruption pf classroom programs and to allow^he unencum- 
bered implementation of the program. ^ In order to strike a baland^ between the 
need for information and the desire to minimize disruption, a strartified random 
sampling procedure was used to gather data on all* the field test objectives 
except the one dealing with standardized testing. . • 

Three monitoring visits were conducted between November 1972 and May 1973. 
Schools were randomly selected from each of the two school categories: cpnyen- 
tionally organized, urban; and multiunit, noniirban. The selection proportion 
was .5 and there was replacement of the selected schools. Interviews for gather- 
ing implementation data were conducted with the people most likely to provide 
the information- One class for providing data about the mastery level on specific 
program objectives ^'^3 randomly selected from all the eligible classes arid another 
class was randomly selected to provide data about the maintenance of this mastery 
level of the specific program obj^ectives. Six students iri the -delected classes ^ 
were randomly chosen to be tested on the appropriate inventory. 

Topic inventories were chos^^n en the basis of the progresS of the class in 
the program* For assessing the mastery level of the students^ the topic inven- 
tory for the last completed topic was used. For assessing the maintenance of 
the mastery level, , the topic iitventory for the next to last completed topic 
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was used. If the topic inventory selected for assessing mastery after • 
instruction had been given already, it was not given again. Instead the 
data used were copied from the teacher's records. 

Data on maintenance of the mastery levels were gathered in only the 
last two monitoring visits. The progress through the second grade materials 
did not allow the proper amount of instruction to occur for assessing this 
field test objective in the first monitoring visit. 

To compare achievement of students using DMP with students not using 
DMP, as measured oh standardized tests, baseline data were gathered in the 
field test schools in the spring of 1972 from second-grade student^ who had 
used the schools* regular mathematics _^rograms . In the spring of 1973 the 
second-grade students who had used DMP completed different forms of the same 
test. This comparison assumes that the two successive sets of secohd-grade 
students are similar on all pertinent matters except the use of DMP. In 
addition, no direct comparison can be made between specific mathematics 
programs and DMP. Given the level of development of DMP, however, no more 
sophisticated comparisons are warranted. * 

Table 2 contains a summary of the data-gathering information. 




4 



> 



2b 



M 

w 

I 

< 



w 

•H 

W 

iH 
U 



•rH 



=3 

(D 

o 
w 

c 

•rH 



> 

V 

o 
o 



e 

0 

u 

Q 

M-l C/) 

0 (D 
> 

• q; o 

CO -f-i 

S 0 











0 






• H 




CO 




















•H 


CO 




}^ 


c 














CO 


g 


g 










u 




0 




0 




0 




c 


u 


c 


0 




0 


•H 




CO 




a> 


•H 




c 






•H 




a 






0 


q; 




u 








e 


8 



CO 
4J 



H 












CO 






c 






•rH 






•H 






> 


rH 
























U 




CO 




fO 


c 










4-> 


•H 


















fC 




73 


0 




ro 


4-> 








> 




fC 




<D ro 


•rH 


0 








> 


c 


z 


rH 






•H (Ti 








QJ 




*J rH 




c 


>1 


C 








q; 




•H 








Q) 




iH 








? 




QJ 












CO 








q; 


C 


fC 




0 a 








CQ 


iH 


O CO 



c 




w 


0 




q; 




73 


•H 


4J 


q; 


u 




On 


0 


+J 


0 


4J 


c 


rH 


c 


QJ 


q; 


q; 




> 


> 






c 




73 


•H 








CO 


q; 


o 


In 


XI 


•H 




0 


0 









c ro 

QJ CM 



CQ 



73 
C 



Uh QJ Ul 

0 > e 

QJ U 

•H 0 

x: ci< 
o 

< - 
•H 

rH rH 

0 ^ 

CO MH 

rH CO 

0 QJ 

Ci< O Eh 



CO 

c 

QJ 
73 

4J 

CO 



QJ 
> 
-H 

fO in 
U vO 
QJ <7» 
£^ rH 
0 ^ 



CO 
4J 
CO 
QJ 

^ r 
in 

•H 



CQ .>i 
ro U 

73 

C V^ 



73 
0) 

a 

rH 

QJ 
> 

QJ 



H 

XI 
fO 
4J 
C 

:3 

o n 
o 

< QJ 
XI 



0 

4J 





c 






0 






•H 




CO 


^ 






QJ 




/C 


4J 




(0 


•H 






















/ w 


g 












u 




0 








0 


QJ 


c 


u 


> 


0 




•H 


•H 








u> 






c 


•H 




•H 


}H 


a 


+J 


O 


0 


QJ 


CO 


u 


QJ 


QJ 


a* 


e 


Q 




CO . 
QJ ! 
•H 
}H 

0 

4J 

c 

QJ 
> 
C 

•rH 

o 

■rH 







0 


QJ 






4J 


CO 


rH 


O 








•H 


CO 




c 




c 


*J 


•rH 




0 


0 


£ 


CO 


C 


Vh 


QJ 


•H 




QJ 




Vh 




4J 


CO 


mpa 








a 






x5 


O 


5 



>1 

> MH ^ 

0 QJ 

o (i 
C B 

C 4J 

QJ C 

4J QJ 

C 73 

•H D 

^>C0 



CO 

CO 
■H 
> 



c 

•H 

0 

•H 

c 



QJ 



QJ ro 

> 

Z rH 



Q^ 

QJ QJ S 
QJ ? 

Jh 4J 73 

x: QJ c 

Eh X3 fd 



I 

}H 

QJ 

c 

•H 



QJ QJ 

^ rH 

•H D 

fC3 ^3 

C QJ 

c x: 

0 O 

•H CO 

CO ? 

QJ QJ 

a > 



CO 

o 
o 

Q* 

CO 
03 

C 
0 
•H 

4J 

03 

c 

QJ 



ERIC 



21 



V 



III 

RESULTS AND DISCUSSION 

As inculcated in Table 2, the fi^ld test provided data for the four field 
test objectives: determining student mastery on specific program^objectives, * 
comparing the achievement of DMP and non-DMP students on standardized tests, 
determining the maintenance o^f student mastery on specific objectives, and 
d<^cumentinq various aspects of implementation* 

^ STUDENT MASftRY ON SPECIFrC OBJECTIVES 

^ As each student completes a topic, 'it is expected that he will either 
master the objectives of that topic or be making progress toward the mastery 
of them. It is expected that 95 percent of the students in any given group 
will meet this criterion. The topic inventories provide the conditions by 
which a teacher can evaluate the students* level of mastery. R&D Center 
staff used these inventories on randomly selected students to determine the . 
. effectiveness of the program in terms of student mastery of specific objectives. 
In the first year ol the field test the classes completed only cibout 70 
percent of the material that was intended for their grade ;Level. Thus, in the. 
second year of the' field test some time was devoted to corrpleting the material 
originally intended for the first year. In addition ,' the first topic in the 
second year 'material contains instruction 'that is utilized in sxabsequent parts . 
of the program and is not asselsed until then. Therefore, the objectives of 
only three topics were assessed in the course of the second year field test. 

■ " These thr^ea topics span seven of seventeen topics considered "usual" for the 

students. In the end-of~year report from all of the field test schools, the 
average ^arly progress through the program was seven and-one-half topics for 
nonurban schools and four topics for urban schools. Table 3 contains the 
percents of ratings of student performance that were either Mastery (M) or 

/ Making Progress (P) on the objectives *in the three topics with assessed 

objectives. 

The percentages reported in Table 3 are below the expected figures, 
' partictilarly in the topic deaJLing^with equalizing situations. It is encourag- 
ing tp /lote that in the nonurban schools the percentages in the number sentences 
topic are much higher than those in the first topic. ^ Thefee two topics are closely 
related and the increased percentages indicate a cumulative learning effect. 

Table 4 contains the percents of each type of rating for the specific 
objectives of the three topics. The figures in Table 3 and Table 4 are directly 
comparable because the mastery criterion reported in Table 3 considers combina- 
tions of ^ Mastery (M) and Making Progress (P) ratings. The percentages in Table 
4 are included here to indicate" any shift in the proportion of M artd P ratings. 
In the two related topics mentioned previously (equalizing situations and num- 
ber sentences) , not ^ly did the percent of nonurlDan students meetina the 
• Mastery criterion increase (see Table 3) but the percent of M ratings overall^ 

increased even more dramatically. 
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To surnmarize the data related to the students* mastery of specific 
* objectives, . the overall percentage of students meeting the criterion was* 
lower than that expected. In ' two of the three topics tested (grouping and 
number sentences) , the percentages were reasonably close to the expected 
figure. Two other results were encouraging: (1) The mastery data for the 
second of two closely related topics is considerably more positive than the 
data for the first topic, which indicates a cumulative effect. (2) There 
was evidence that lirban and nonurban students are attaining similar mastery 
levels for the same topics. Urban students did not progress as quickly through 
t the program, but they attained essentially the same levels of mastery. 

: . ✓ 

COMPARISON OF DMP STUDENTS WITH NON-DMP STUDENTS* 

* The second field test oj^jective is to compare the achievement of DMP and 
non-DMP students on standardized tests. The mathematics subtests of the 
California Achievement Test and the Cooperative Primary Test were used to 
assess student .acjiievement. ^In May 1972, baseline data were gathered for 
second-grade students in field test schools who had not participated m the 
program. In May 1973, the alternate forms of the same tests were given to 
students who had been using the program for two years but who had used conven- 
tional programs as kindergarteners. The above 'procedure does not account for 
students who transferred into the schools during the field test. Schools with 
a high turnover in student populations will have many students who do not have 
as much experience with DMP as .those in other field test . schools . 

Table 5 contains comparative data on the two administrations of the standard 
ized tests. The data are reported as percentiles in order to facilitate compari- 
son of the results, ajhe ^percentiles were derived from the means of the school 
raw scores means. * * ^ 

As the data in Table 5 demonstrate, the DMP students in nonurban field test 
schools generally show lower achievement levels than comparable non-DMP students. 
For students in urban schools, two of the four comparisons favor the DMP students 
These results are not entirely surprising nor discouraging, for DMP is designed 
to develop conventional arithmetic skills only after a careful development of the 
conceptual basis for the skills. A consequence of this design is that mastery 
of some arithmetic skills in DMP instruction is expected at a later date than 
in other mathematic^ programs. Standardized tests typically reflect the instruc- 
tional patre of conventional mathematics programs, putting DMP students at a dis- 
advantage. This disadvantage was compounded by the relatively slow progress of 
the field test students through the program. Therefore, the .differences between 
the scores of DMP students and the comparable non-DMP students are well within 
acceptable limits. It is anticipated that the differences will vanish, and m 
fact reverse themselves to the advantage of DMP students, in the latter stages 
of the program. 

A more sensitive analysis of the standardi;2ed tests can be accomplished 
by considering only those items that are related to DMP Instructioh as recom- 
mended 1^, Romberg and Montgomery (1971). Each item was inspected for its 
appropriateness to specific objectives in the program. Those items that can 
be "equated with specific objectives were analyzed to determine the program's 
efficiency in terms of the student's learning, given his opportunity to learn. 
For example, students who have progressed to a given point in the program pan 
be expected to answer correctly all items that have been equated with objectives 
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TAfeLE 5 



PERCENTILE EQUIVALENTS OF MEAN* RAW SCORES FOR TWO ADMINISTRATIONS 
OF THE MATHEMATICS SUBTESTS OF TWO STANDARDIZED 4ACHIEVEMENT TESTS 







School 


Type 






Urban * 


Nonurban 


Subtest 


May 1972 


May 1973- 


May 1972 


May 1973 


California Achievement Test 


Fonn IB 


Form lA 


Form IB 


Form lA 
* 


Computation 


20 


18 


65 


29 


Concepts £ Proble^te 


23 


35 


75 


54 


Total Mathematics 


20 


24 


76 


X 42 ' 


Cooperative Priniary Test 


•Form 236 


Form 23A 


Form 23B 


Form 23A 


Mathematics 


22 


16 


66 


61 



contained in the program to that point. The degree to which that expectation 
is met is the degree of efficiency of the* prograun.- 

This procedure developed out of the analysis of the data collected for 
the Internation^^i^Study of Educational Achievement (Husin, 1967). As applied ^ 
to the DMP field test data, the procedure yields a fair comparative assessment 
of the effectiveness of DMP. - ^ 

In equating the test items to the DMP objectives, two levels of similarity 
were noted: (1) some items were stated in such a way that a student who masters 
the DMP objective is expected to answer the item correctly; and (2) some items 
were stated in such a way that the item content is similar to a DMP pbjective 
but not similar enough to expect the student to answer the item correctly. In 
addition, some items were found for which no DMP objective could be equated. 

Another factor to be considered in calculating the efficiency ratios was 
the progress of the students .through the program. Only those equated items for 
which there had been instruction could be included in the analysis. Therefore, 
an addition tto the two categories for the two levels of item similarity, and 
the one category for those items ^not related to a DMP objective, there was a 
set of 'items , that were similar to DMP objectives but could not he included in 
the calculation of efficiency ratios because the instruction had not progressed 
far enough to include tl^e equated DMP objectives. Table 6 contains the propor- 
tions of items in each ^ category . 

The data in Table- 6 show that the field test schools had progressed to the 
point where as much as 55 percent of the itepis were related to the instruction 
(for nonurban students on' the California Achievement Test) and as few as 19 
peioent of them were related (for urban students on the Cooperative Primary 
Test) . If the field test schools had progressed^t a rate near the one expected 
by the developers, more than' 45 percent, of items m the Cooperative Primary 
Test and 80 percent of those in the California Achievement Test would have 
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TABLE 6 

MEAN PROPORTIONS OF EACH- OF FOUR SUBSETS ^QF ITEMS FOR^ TVi70 ACHIEVE'mENT 
TESTS GIVEN ^ TO DMP STUDENTS IN TWO TYPES OF ^SCHOOLS " 



■ / 



— ^ 

-J ' 

* Number 
School Type/Test of Items 




Item 


Subset 


* 


Equal 


Content 
Similar 


^pvond 
Progress 


Unrelated 


Urban 












California Achievement Test 


87 


.13 


< 

.07 


.80 


.00 


Cooperative Primary Test 


6'0 


.07 


, .12 , 


^ .73 


.08 


Nonurban 
























California Achievement Test 


87 • 


.46 


.09 


.40 


.05 


Cooperative Primary Test 


60 


.12 


.20 


.60 


.08 


Overall 


i 
1 
I 










California Achievement Test 


87 


.30 


.08 


.60 


.02 


Cooperative Primary Test 


60 ! 

1 


- .09 


.16 


.67 


.08 



been related. When the students complete the program, over 90 percent of the 
items will be related to the content of DMP. * ' 

The efficiency ratios for ea.ch of the related-item subsets are shown in ^ 
Table 7. ' 

A column of comparative ratios is included to assist in the interpretation 
of the efficiency ratios. The comparative ratios reflect the proportion of cor- 
rect items that are peeded for a student to attain the 50th percentile of a 
given test. The data clearly show that tlfie field test students are answering 
^ the items correc^tly at the appropriate level when the progress of the instruction 
is considered. An exception .to this is the* urban students on the content-similar 
items. Although the content of each of these items was similar to a DMP objective, 
the similarity was not strong enough to assume a carrect response from students 
who had mastered the objective. The urban students did not perform as well on 
items from this category as they did on items where the relationship between 
the item and the objective is more exact. 

In orSer to set another context for interpreting the efficiency ratios 
presented in Table 7, a similar analysis was performed on the baseliae data 
gathered in May 1972 from students not using DMP. Strictly speaking, trfis'~~~ 
analysis did not yield efficiency ratios because the developers of the jdathe- 
matics programs uped by non-DMP students did not equate the test items with their 
instrucLiunai c]jjt;ctivcG . In^btead, the test items were related to DMP objectives 
as in the. previous analysis. The resulting ratios indicated the degree to which 
stude.-.tc who do not use DMP were able to learn material covered by DMP. Not 
all of the DMP objectives were considered in this analysis nor in the first 
one. Only those DMP <^bjectives that are related to test items were considered. 
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r ' TABLE 7 

EFFICIENCY PATIOS. FOR TWO SUBSE^TS OF RELATED ITEMS 
ON TWO ACHIEVEMENT TESTS TAKEN BY DMP STUDENTS 
IN TWO TYPES, OF 'SCHOOLS ' ^ * 

(^Y 1973) * • ' ' 



N N Content \ N 

School Type/Test Items Equal Items Similar Items Total Comparative 

Urban . * ^ * ' 

' -California Achievement' 11 .883 6 .,66^5 - 17 W -606 - .805' 

Cooperative Primary ^ 4 .506 ' 7 .4Q6 11 .481 .500 

Nonurban 

California Achievement 40 , .908 8 : .925 48 .911, .897 

Cooperative Primary* - 7 . .685 ^12 .fe38 19 .655 .500 

Overall* - * . 

^California Achievement- ^ .896 ^ .795 . .859 ^ .851 . 

Cooperative Primalry .596 .552 ^ .557 -500 

* Unweighted mean- - ^ ' 

The comparative .ratios presented in Table 8 were calculated in the same 
manner as the efficiency ratios presented in Table 7. The items were categorized 
according^o their similarity to DMP objectives. The same progress through DMP 
was assumed when determining which items to include in the analysis. 

The ratios in the two tables are remarkably similar. At Grade 2, DMP and*' 
the mathematics programs used previously in the fi^ld test schools are about 
equally effective. in their instruction on items selected for their similarity 
in* content to. DMP objectives. 

The efficiency ratios for the two remaining categories of items are reported 
in^Table 9 to denote the decline in the students* achievement levels. It is 
interesting to note/ hoy^ver / that the students attained the 50th percentile 
on the urJrelated items of the Cooperative Primary Test. Apparently ' students 
n adequately learn this* material outside of DMP instruction. 

Table 10 contains the comparative ratios for the last two subsets of 
items taken from the baseline data« of students not using DMP. These ratios 
^ow the areas in which the nonurban baseline students outperforined the DMR 
students. 

To summarize the results of the comparison of students* achievement on 
standardized tests, there is no conclusive evidence that favors either DMP 
or those, programs that the field test schools used previously. The scores 
on the California Achievement Test f or (^lonUrban students appreriably favored 
the conventional programs. This' was not the case for urban students on that 
test. Scores on the Cooperative Primary Test for both school types only mildly 
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TABLE 8 

COMPARATIVE RATIOS FOR TOO SUBSETS OF R£LATED ITEMS 
OF TWO ACHIEVEMENT TESTS GIVEN TO STUDENTS NOT USING DMP 
IN. TWO SCHOOL TYPES*.. 
(MAY 1972) 



Schcx)! Type/Tes*- 



, N N N ' 

Items Equal Items Similar^ Items Total Comparative 



Urban 

California Achieyenient • 11 
cooperative Primary 4 

Nonurban 

California Achievement 35 
Cooperative' Primary 7 

0</erall* . " 

California Achievement- 
Cooperative Primary 



.880 
.404 

.944 

.668 

.912 



5 
7 

.9 
9 



,609 
.583 



17 .784 
ll' .518 



,850 44 .925 

.705 16 .689 



.729 
,644 



-.854 
.604 



Unweighted means. 



TABLE 9 



EFFICIEN^CY RATIOS FOR TWO SUBSETS OF UNRELATED ITEMS 
OF TWO ACHIEVEMENT TESTS GIVHN TO DMP STUDENTS 
IN TWO SCHOOL TYPES 

(MAY 1973) ^ 



.805 
.500 

.805 
.500 

.805 
.500 



School lype/Tesc 



^ N Beyond N N . Compar- 

Items Progress Item^ Unrelated Items Total ative 



Urban 

California Achievement 70 .521 

Cooperative Primary ^44 . 349 

Nohurban . . ' » ' 

California Achievement 35 .797 ' 

Cooperative Primary 36 . 519 

Overall* 

California Achievement .659 

Cooperative PriAiary .434 



.496 

.695 
.513 

.695 
.504 



70 
49. 

39 
41 



.521 .80S 
.364 .500 



.787 
.518 

.654 
.441 



,897 
.500 

.851 
' .500 



* Unweighted means* 



22 



I 

4* 



topic prevented the gathering of sufficient data from which to generalize. 
Tests were given for only two topics (Symmetry, Fractions, and Shape; and 
'Representing Joining and Separating Situations), no tests could be given in 
ui:ban schools. The two topics that were tested were not tested as a part of 
mastery of specific cJbjectives and hence no direct comparisons can be made 
between the mastery of objectives and their subsequent maintenance. In sub- 
sequent years of the field test more complete data can be gathered. 

Table 11 contains the proportion of students who attained the mastery 
criterion, Th'e same criterion was used for these data as the one for the ^ N 

mastery of specific objectives: receiving a Mastery (M) or Making Progress (P) 
rating on the objectiives of the topic. The data indicate, relatively high levels 
of mastery. 



TABLE 11 



MEAN PERCENT OF M OR P RATINGS* OF STUDENT PERFORMANCE 
ONE MONTH OR MORE AFTER INSTRUCTION ON THE OBJECTI\^S 

OF jCWO topics^ 



School/Type 


Symmpt-ry . Fractions , 
and Shape 


Representing Joining' 
and Separating Situations 


Total 


Urban 








Nonurban 


83 


83 


.83 


* 

Mastery (M) 
, • * 


or Making Progress (P) on 


each objective, 

• 





Table. 12 contains the percents of each type of rating. These proportions 
.arc directly comparable to the fiVuie:^ contained in Table 11, The percents in 
.Table 12" ind;Lcate > slight increase m the mastery ratings in the second topic, 

but this is hot an important result since the topics are not in the same content 

strand. 



' TABLii 12 

MEAN PERCENT OF EACH TYPE OF RATING* OF STUDENT PERFORMANCE. 
• ONE. MONTH OJ^ MORE AFTER INSTRUCTION ON THE OBJECTIVES 

OF TWO TOPICS 



Symmetry, Fractions , Representing Joining 
School/Type and Shape and Separating situations Total 



Urban 



M , 37 - 50 44 



Nonurban P • 46 



3o ' .39 



N ' 17 ♦ ^ 17 . 17 

Mastery (M) , Making Progress (P) , Need.s Considerable Help (N) , 
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topic prevented the gathering of sufficient data from which to generalize. 
Tests were given for only two topics (Symmetry, Fractions, and Shape; and 
" Represeriting Joining and Separating Situations). No tests could be given in 
ui:ban schools. The two topics that were tested were not tested as a part of 
mastery of specific objectives and hence no direct comparisons can be made 
between the mastery of objectives and their subsequent maintenance. In sub- 
sequent years of the field test more complete data can be gathered. 

Table 11 contains the proportion of ctudents who attained the mastery 
criterion. The same criterion was used for these data as the one for the \ 
mastery of specific objectives: receiving a Mastery (M) or Making Progress (P) 
rating on the objectives of the topic. The data indicate relatively high levels 
of mastery. 

TABLE 11 

MEAN PERCENT OF M OR P RATINGS* OF STUDENT PERFORMANCE 
ONE MONTH OR MORE AFTER INSTRUCTION ON THE OBJECTI\^S 

OF lyO TOPICS^ 



School/Type 


Symmpi-ry. Fractions , 
and Shape 


Repiesenting 
and Separating 


Joining * 
Situations 


Total 


Urban 
Nonurban 


* 

83 


83 






* 

Mastery (M) 


or Making Progress (P) on 


each objective. 







Table. 12 contains the percents of each type of rating. These proportions 
,2rc directly comparable to the ficrure^^ contained in Table 11. The percents in 
, Table 12 ind;Lcate> slight increase in the mastery ratings in the second topic, 

but this is hot an important result since the topics are not in the same content 

strand. 

'* . TABLii 12 



MEAN PERCENT OF EACH TYPE OF RATING* OF STUDENT PERFORMANCE, 
ONE MONTH OP MORE AFTER INSTRUCTION ON THE OBJECTIVES 

OF TWO TOPICS 



School/Type 




Symmetry, Fractions , 
and Shape 


Representing 
and Separating 


Joining 
Situations 


Total 


Urban 












» 


M 


. 37 


50 




44 


Nonurban 


P 


46 


33 ' 




39 




N 


17 


17 




17 



* Mastery (M) , Making Progress (P) , Needjs Considerable Help (N) . 
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IMPLEMENTATION CONSIDERATIONS * , ^ 

The fourth field test objective deals with implementation information 
that* IS important to school personnel. This information concerns prograri\ 
costs; staff responsibilities, time anocation, and material usability. * 
Some of the mfonnation reported m this 'aection deals with follow-up data 
for the fj.rst two levels of the program. 

Initial implementation costs of the second-grade materials ranged from . 
$6.50 to ?10.uu per student, depending on the number of classes that shared 
the niaterials kit. .Most multiunit schools spent somewhat less per student 
than conventionally-organized schools. However, .^in general the initial cost 
of the second-grade "package" is somewhat more than that for either Level 1^ 
or Level 2 since there is a single materials kit but there are two different 
workbooks, test. booklets , etc. 'Because of their field test status, the schools 
themselves actually U^re from .40-45 percent of the per-pupil cost (materials 
kit), while the Centet absorbed the remaining amount (the printed materials). 
Since the materials kit is all-mclusive , containing even such unspecialized 
Items as rubber bands, yardsticks, and yarn, no other expenditures beyond those 
for ordinary ci.assroom supplies were, necessary. 

Actual ger-student costs for continuing the first two levels of the program 
ranged from ^$1.40 to $1.83 and averag<=d $1.57. These figures included costs for 
changing versions of teacher materials which will not be necessary when the prograr 
5/S finally implemented. Theoretically the continuing 'costs (including the costs 
fof the new teacher materials) si^uld be $1.35 per, student when calculated on 
the basis of 32 pupils per classroom. The average actual costs for continuing 
the program were l6 percent higher than the theoretical costs. 

The above figures were the costs fpr the Afield test version. The publisher 
has upcfated these figures to reflect tVieoretical commercial, costs . Those 
theoretical costs are reported in Table 13. For comparison purposes the costs 
of J:hree other mathematics programs. are included. The other programs were 
selected as widely representative of proyi:ams that are used by schools. One 
is a standard text-based program, the second is a popular text-based program 
that offers a considerable number of enrichment activities and an assessment 
package, and the third is a combination of innovative programs that is the most 
comparable to DMP in that it includes a text, an , activities package, and an 
assessment package. 

TABLE 13 

COMPARABLE PER-STUDENT COSTS OF DMP AND THREE OTHER MATHEMATICS 

PROGRAMS FOR GRADES K-2 ' 



Program First Year ' Continuation Over Four Years 



DMP 




9.63 


. 4 


2 


16 


4, 


03 


*Standard 
















Cloth-bound Text 




5.05' 




2. 


11 


2 


85 


Consumable Text 




3.95 




3 


61 


3. 


70 


♦Popular 




5.9-0 




2 


20 


3. 


13 


♦Combination 




8.82 




3 


79 


5 


05 



See text for description. 
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The jcosts for DMP compare favorably to the programs listed above. Over 
a four-year period, DMP is slightly more expensive than the consiimable ver- 
sion of the standard text series and cheaper than the new program that com- 
pares most closely to DMP. The popular textbook series and-the cloth-bound 
version of the standard text series are less expensive than DMP but, even 
with enrichment activities and an assessment program, they do not offer ^s 
complete a program as DMP. 

Inservice time, planning time, and the responsibilies of ^dded staff, 
such as aides, were also considered. Continuing. teachers who had received 
inservice training in the first year of the field test did not require further 
inservice. They also reported spending somewhat less time preparing for DMP 
in their secon'd year due to their experience with the program. New teache^rs 
were requested to attend a one-day workshop in the fall, bimonthly hour- long 
meetings for the first semester, and a half-day mid-year workshop. These in- 
service days were well-attended and well-received. However, it is. the Center's 
judgment that the new teachers required less inservice guidance during the 
school year than new teachers during the first year of the fiel^ test (probablv 
due to the presence of experienced DwP teachers in each bull ding). Also, al- 
though the new teachers still found it necessary to spe^d a considerable amount 
of tiine preparing for DMP, in general there were fewer complaints in this area, 
and it is possible chat less time was actually spent, due to program improve- 
ments. With the change tc workbooks and a comprehensive materials kit, time 
spent in the previous year duplicating worksheets or searching for materials 
could be applied to other classroom preparations. Also, the students came to 
the teachers with a DMP background, a factor that may have eased the teachers* 
adjustment. Although ther^ was evidence of reductions in additional staff 
time, DMP cannot be a conventionally- taught program, and a commitment to 
extensive preparation and inservice time still must be a prerequisite for 
program adoption. 

Where available, teacher aides were again used extensively and were viewed 
as invaluable. However, as in t^ie first year of the field test, teachers in 
Chicago successfully used DMP without the benefit o'r aides, where aides were 
employed, they prepared materials, set up stations, assessed students, kept 
records, and assisted with instruction. No change in the amount of aide time 
utilized for DMP was ndted from the first to the second year, but there" werp 
shifts in services performed. Since there were mor*e ^aper-and-pencil tests 
available in 1-Kp second year, some aides participated in more assessment activi- 
ties. Aides, were utilized more for instructional assistance, due very likely to 
fewer time-consuming duplicating tasks. 

Appropriateness, or usability of materials, for students and teachers was 
examined with particular attention to how changes in the materials were received 
by continuing teachers. General attitudes of new and continuing students and 
teachers were also surveyed*! Further, specific errors or confusion in the 
instructional materials were documented and appear in Appendix C; the formative 
reports were purposely concentrated on the second grade materials, which were 
in their first year of field testing. All reports were submitted to the 
rJevelopers for consideration of revisions. Except for isolated comments about 
specific activities, almost all program revisions were warmly received by the 
continuing teachers. Prominent among t^e most welcome changes were the addi- 
tion of all-inclusive materials kits and the new balance beams. The workbooks 
we're generally approved, especially since 'page perforations made it easy to be 
flexible in using the workbooks. A minority of teachers continued to express 
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the feeling that workbooks lead to a lockstep approach and lack of individ- 
ualization. Tiiere was also a fairly continuous request for a separate group 
of workbook activities, or even a separate workbook, games for the childpn 
to play independently, drill-and-practico exercises, etc. Among the ma3or 
positive modifications in the teacher materials was the new Topic Sequence 
Chart and the repetition of tht objectives throughout* the materials for easy 
reference. Appreciation was expressed for brevity in activity explanations; 
however, there were still some complaints about "too much verbiage." ^ 

Revisions in the assessment component:, such as new paper-and-pencil tests 
and shortened tests ,j^were looked upon favorably. Teachers had difficulty using 
placement inventory results to group students for instruction in specific 
objectives or topics. They did, however, find the results infonnational and 
useful for general group background and for roughly grouping a large number 
of students into two or three small groups. There remains a general lack of, 
* confidence in these inventories. The record forms were not revised sub- 

stantially and, as in the first year, teachers chose to use one of the three 
record cards., usually the Group Record Card. The Topic Checklists were used 
more and more infrequently; this seems to be directly related to the greater 
availability of paper-and-pencil tests, especially at Level 3. There was a 
strong tendency to avoid making formal observations and to simply assess all 
students on the paper-and-pencil tests - 

The new teachers had few negative comments concerning the usability of 
the second-grade materials, which of course incorporated the revisions 
Le'^^els 1 and 2 discussed above. Thje single outstanding problem c^airie with 
' the first two topics m €ne series of sentence topics, 3.1 and 3\3. They 

were exceedingly difficult to pace and were confusing to both teachers and 
students. When the students could not demonstrate the desired level of 
mastery, teachers ten<fed to bog down, reinstructing the students in these 
topics. Further detailed comments appear in Appendix C. Only isolated 
classes reached Level 4, so usability information is not . available . 

Teachers, whether new or continuing, were again very dissatisf x/sd with 
the packaging of all levels of materials kits. They repeatedly suggested that 
all items be well-labeled and have sturdy containers in which to be organized, 
stored, and transported. 

Perh^aps the greatest problem concerning the usability of the program was 
the students^ lack of progress through the program. The slow pace was notice- 
able during the first field test year but was attributed to the initial* problems 
with the adoption of any new program. During the second year, however, it vj^s 
increasingly obvious that students were simply moving very slowly through the 
prog2>am. Only^two of the twenty-five Grade 2 groups had moved into Level 4 
by the end of the year. ' A variety of factors seem to be responsible. Many 
groups had not completed Level 2 topics. There were problems with early 
topics in Level 3 (discussed above). Teachers tended to complete nearlv ^ 
every activity offered even though it was not presctibed m the Teacher^ s Guide. 
Whatever the reasons, at that pace the field Lest bcuaenc? were not iiKeiy. . 
to complete a reasonable number of DMP levels before the end of elementary f 
school. 

In seven of the eight schools, the attitudes of pupils and staff toward 
DMP were ov^rwhelrriingly positive. Much of the positive teacher response to 
the program seemb to be , a result of student enjoyment and success at lea«:ning' 

— with the exception of the early topics in Level 3. In the school in 
which the attitude became negative, the major objection was the discrepancy 
between instruction m DMP and standardized test objectives. 

« 
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SUMMARY 



The continuation ot'a field test begun in 1971-72 was conducted in the 

second grade of the sainfe. ei^Jht schools. Four were multiunit schools located 

m 'Settings that ranged krom small towns to large cities; four were conven-" , 
tionally organized and located in large urban areas, xne purposes of the 
fieia test were (1) to determine the effectiveness of the J^nstructional program 
in terms of the achievement of the students, and (2) to document the usabili|:y 
of the program. 

The effectiveness of the instructional program was assessed by administer- 
ing program tests to random samples of studehts. 'The testing sessions were 
conducted during tnree monitoring visits made to stratified random samples 
of field test schiads." The tests were selected according to one of two criteria: 
appropriateness for assessing mastery of the most recent instruc^tional objectives 
or appropriateness for assessing mastery of those objectives taught just before 
the most recent instructional unit. These data provided infopiati9n on the 
mastery levels of the students and their maintenance >of those mastery levels. 
The effectiveness of the program was also assessed by comparing t^ie results 
from standardized tests given to DMP students and nomJDMP students. 

The results of these investigations were generally satisfactory. Approxi- 
mately 81 percent of the students attai.ned the specified criterion of^ Ifestery 
(M) or Making Progress (P) . This percentage was hot uniformly obtained on ?11; 
objectives; the range was 54 percent to 100 percent. T^diff^ences seetn^^ 
to indicate a need to revise the materials. When studdB^, W^re '/a^&essea^for ^ -f^^ 
maintenance of mastery levels, approximately 83 percent *of the students were 
at the expected level of mastery (Mastery or Making Progress). These effects 
were not firmly established due to a weakness in the collection of the data./ ^"-^ 

The results of the comparisor;is of standardized test scores were mixed. 
For the multiunit schools there was a noticeable defici^^cy in zhe scores of 
the DMP students, particularly on one cf the two tests used.^ For the urban 
schools the scores generally favored. the DMP students. In an item analysis 
that accounted for the progress of the DMP students through the program,^ the 
achievement level was found to* be above the expected national norms fOL both 
the urban and nonurban students. .*. ' / .0 

The usability of the program was well docuinented. The cdmmercial cost ; ^ 
was' found to be competitive with a range of existing mathematics programs. 
Although the requirements of additional preparation time and inservice time 
still existed, th^^re was some evidence of a reduced need through improvements , 
m the materials 'and familiarity with the program in the second year of the ' ^ 
fjk$ld test. A few of the topics were found to be troublesome' to implement, . 
but in general the enthusiasm for the. program was sustained through the second 
year. Finally, it was found that the problem of pacing the students through;^ 
the^program was not due solely to the fact that. this was a field test. . r 

This secondK ield test renewed the hopeful note which ended the report^ 
of i'he first field test. Teachers wi 11 expend the effort to adapt to the .^V^"", 
program. Students will master the objectives and they both will be enth^^i^^^-^; 
tiC about the endeavor* Av7\^ 
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Appendix A 
Memoranda of Agreement 



MEMORANDUM OF AGREEMENT , 
between 

The Wisconsin Research and Development Center for Cognitive Learning 

and 

Milwaukee School District 

The Wisconsin Research and Development Center for Cognitive Learning (Center) 
and the Milwaukee School District (District) agree cooperatively to field test 
during the 1972-73 academic year Levels 2 through 5 of Developing Mathematical 
Processes (DMP) instructional materials which were developed by the Center. 
The fiela test will be conductedj^at Fourth Street School with all children 
in their second through fourth years of school and at Victory School with 
all children m, their second 'and third years of school. The Center agrees 
to advise its vendor, Rand .McNaliy, tg fill orders for materials placed by 
the school district. 

A. The Center will provide at no cost to the District: 

1. One day of staff inservxce prior to the opening of -school and one 
half-day of staff inservice mid-year. This * inservice is designed 
for teachers new to DiMP regardless of grade level. However, con- 
tinuing teachers are welcome to attend. Participating teachers 
and aiSes will be reimbursed at their usual hourly rate for such 
inserv:j.ce. 

2. All field test materials for teachers and all printed instructional 
field test materials for children for Levels 3-5. If the district 
purchases revised activity and test booklets for Levels 1 and 2, the 
Center will provide all^ associated printed materials for teachers. 

3. Criterion-referenced tests directly associaSed with the instructional 
program for Leyels 3-5 and tests associated with the gathering of 
criterion data. 

4. Feedback to school systems regarding the field test results in the • 
form of a written report. The initial report will be provided by 
August 20, 1973, with a more extensive report to follow by December 

* 31, 197*3. 

5. Feedback -and assistance to the staff of the participating school 
at least twice a semester in the form of a half-day visit by a 
Center staff person. 
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6. A non-professional person (aide) fur three hours per day to assist 
the cooperating teachers in Fourth Street School in implementing 
the program. 



B. The District will insure that the participating schools will: 

1. Provide the manipulative aids required for program implementation 
per Center recommendations. 

2. Engage all eligible pupils and staff in the participating school (s) 
in the program, as indicated in Paragraph One of this Memorandum of 
Agreement. 

3. Pay any shipping costs for returning tests to the Center. 

4. Devote a minimum of two and one-half hours instructional effort weekly 
per child to the DMP program. 

5. Pr-ovide up to two hours of pupil time for the gathering of criterion 
data yearly; apprise the Center of the schedule anu procedures of 
the local testing program; and share with the Center any intelligence 
or achievement data from the participating schools gathered through- 
out the District's testing program. 

6. Inform the Center upon Board of School Directors* attention to and 
approval of any school boundary changes affecting over 10 percent 

of the enrollment of a given school, so that termination of the test 
at the affected grade levels can be jointly considered. 



C. It is furthermore understood that the 1972-73 school year is the second 
year of the DMP field test, and that the Center and school anticipate con- 
tinuation of the test in at least the 1973-74 school year and for children 
in their fifth and sixth years of school, should both parties agree that 
the second year of the test is successful. 

» 

Agreed to: \ Agreed to: 

r 

William R. Push, Deputy Director 
Wisconsin Research and Development 
* Center for Cognitive * Learning 



(Signed) 



(Title) 



ERIC 



(District) 



(Date) (Date) 
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MEMORANDUM OF AGREEMENT 
between 

The Wisconsin Research and Development Center for Cognitive Learning 

•and , 

The Chicago Public Schools 

,The Wisconsin Research and Development Center for Cognitive Learning (Center) 
_an^ the Chicago Public Schools (District) agree cooperatively to field test 
"during the 1972-73 academic year Levels 2 through 5 of Developing Ma'thematical 
Processes (DMP) which were developed by the Center. The field test will be 
conducted in Parkman and Hair-line School (s) with all c}}ildren in their second 
through fourth years of school and in Willard School with children in their 
second and third ye^rs of schools The Center agrees to advise its vendor, 
Rand McNally, to fill orders for materials placed by the school district. 

A. The Center will provide at no cost to the District: 

1. One day of staff inseirVice prior to the opening of school and one 
half-day of 'inservice mid-year.^ Th'is inservice is designed for 
teachers new to DMP regardless of grade level. However, continuing 
teachers are welcome to attend. 

2. All field test materials for teachers and- all printed instructional 
field test materials for children for Levels 3-5. If the district 
purchases revised activity and test booklets for Levels 1 and 2, the 
Center will provide all associated printed materials for teachers.- 

3. Criterion-referenced tests directly associated with the instructional 
program for Levels 3-5 and tests associated with the gathering of 
criterion data. ' ^ 

4. Feedback to school systems regarding the field test results in the 
^ form of a written report. The initial report will be provided by 

August 20, 1973, with a more extensive report to follow, by December 
31, 1973. 

5. A professional part-time (40%) person to assist the cooperating 
teachers in Jonplementing the program. 

t 

6. Feedback and assistance to the cooperating staff of each sc'hool in the 
form of a visit at least twice a month. Ordinarily this visit will be 
made by the person provided under A. 5. 
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The District will insure that the participating school (s) will: 

1. Pro^^ide the manipulative aids required for progrcim implementation 
per Center recommendations. 

2. Engage all elig^ible second through fourth year pupils and staf f \n 
the participating school (s) in the program. 

3. Pay any shipping costs for returning teats to the Center. 

4. Devote a minimum of two and one-half hours instructional effort 
weekly per child to the DMP program. 

5. Provide up to two hours of pupil time for the gathering of criterion 
data yearly; apprise, the Cer.tcr of the schedule and procedures of 
the local testing program; and share with the Center any intelligence 
or achievement data from the participating schools gathered throughout 
the District's testing program. 

6. Inform the Center in advance of school boundary changes affecting 
over 10% of the enrollment of a given school, ^ that termination 
of the test at the affected graue levels can be jointly considered. 



It is furthermore understood that the 1972-73 school year is the second 
year of the DMP field test, and that the Center and District anticipate 
continuation'^of the field test in at least the 1973-74 school year and 
at Grades 4 and 5, should both parties agree that the second year of the 
test is successful. 



Agreed to: 



Agreed to: 



William R. Bush, Deputy Director 
Wisconsin Research and Development 
Center for Cognitive beaming 



(Signed) 



(Title) 



(Date) 



(District) 



(Date) 
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MEMORANDUM OF AGREEMENT 

^ between 

The Wisconsin Research and Development Center for Cognitive Learning 
• * * • 

* and^ / ^ . ' * 

' ' . * .' * ^ 

(^reen Ba^, Gale - Ettrick-Trempealeau , Sparta Area Public SchcK)ls 

'The Wisconsin Research and Development Center for Cognitive Learning (Ce^iter) 

—i : : (District) 

agree cooperatively to field test during the 1972-73 academic year Levels 2 
through 5 of Developing Mathematical Processes (DMP) which were developed by 
the Center* The fd,eld test will be conducted in 

« School (s) with all children in their second * 

through fourth ^ years in school. The Cefiter agrees to advise its vendor. Rand 
McNally^ to fill orders for materials placed by the school district.' 

A. The Center will provide at no cost to the District: ' 

1- One day of staff inservice prior to the opening of school and one half-' 
^ day of inservice mid-year. This inservice is designed for teacher;s new 

to DMP regardless of grade level. However, continuing teachers are 
welcome to a.ttend. 

2. 'All field test materials for teachers and all printed instructional 
field test materials for children for Levels 2-5. If the district 
purchases revised activity and test booklets for Levels 1 and 2, the 
Center will provide all associated printed materials for teachers, 

3- Criteri-on-referenced tests directly associated with the instructional 
program 'for Levels 2-5 and tests associated with the gathering *of 
criterion data. ^ . <s^ • ' 

^4. .•Feedback to school systems regarding the field test results in the 
form of a written report. The initial report will be ^provided by 
August 20, 1973, with a more extensive report to follow by December 
31, 1973. 

* 

5. Feedback and assistance to tjie staff of the participating school at 
least twice a semester in the form of a half-day visit by a Center 
staff person. 
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The District will insure that the participating school (s) will: 

1. Provide the manipulative aids required for program implementation 
per- Center recommendatioas. 

' .« <» , 

2. Engage all eligible second-fourth year pupils and staff in the 
participating school (s) in the .program. , ' ^ ^ 

3. Pay any shipping costs for returning tests to the Cen€.er. 

s • ' • ' • 

4. Devote a minimum of two and oneZ-half hours instructional effort 

weekly per child to the DMP pJbgram. ^ 

< 

5/ Provide up to two' hours of pupil time for the gathering of criterion 
data yearly; apprise the Center of the schedule a^nd procedures of . 
\the local testing program; and share with the Center any intelligence 
or achievement data from the participating schools gathered throughout 
•the District's testing program. 

6. Inform the Center in advance of school^boundary changes affecting 
over 10% of the enrollment "bf a giverWschool, so that termination 
of the test at the affected grade levels can be jointly considered. 



It is furthermore understood that the 1972-73 school year is t^he second 
year of the DMP field test, and that the Center ;^nd District anticipate 
cohtirtuation of the field test in at least the 1973-74 school year and at 
Grades 4-5, 'shbuld bott) parties agree that the second year of the test is 
successful. 



Agreed to: 



Agreed to: • 



William R. Bush, Deputy Director 
Wisconsin Research and Development 
Center -for Cognitive Learning 



(Signed) 



(Ti^le) 



(Date) 



(District) 



(Date). 
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Appendix B 
Interview Forms 
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hSonitoring Guide 
DMP 



PRINCIPAL 

1. What 'costs were involved in contJinuing DMP in K~l? 



2. What is your opinion of the reaction of the teachers, pupils, and parents 
to DMP? 



3. Have you noticed any differences in the teaching approach of the teachers" 
using DMP? 
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^nd GRADE TEACHER 



,1. Have you had problems using DMP this year? (problems with materials, 
directions for teachers, recommended teaching processes) 



2. Have you noticed any advantage to^using DIff this year? (enthusiasm of 
students, more learning, transfer principles to other subjects) 



ERIC 
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Do you use the assessment materials? (Placement Inventory, Observatiuii, 
Topic Inventory, Checklists,, Individual Progress* Sheets) if so; are they 
useful? Any problems? (time, ease of adminucration) .If not, how do * • 
you evaluate the children in DMP? 



4. If the Topic Checklist is used, make a judgment of a proportion of mastery/ 
progressing/non.-masteify for each objective instructed this year. 



/ 
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Did you confer or receive information from last year's teacher regarding 
the student's progress in DMP? If so what and how. did ycu use it? (Look 
^or starting up where thev left off, or using a Placement Inventory, or 
teacher judgment.) 



Do you have groups of students working on different topics at the same 

time? ; Do you have groups of students working at different 

levels within a topic at the same time? How do you form instruc- 

tional groups? i ^ 



Would you do mot^ grouping of different varieties if you could? 4 
If yes, what prevents you? * 
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K OR FIRST GRADE TEACHER 



1. What is your reaction to the revised DMP materials? (improved, about the 
same, worse; effect on teaching) 



2. Is your use of DMP any different this year? (Try to sift out one year's 
experience if possible; i.e., compare fall 1972 to last winter or spring.) 



49 



Are you using the assessment materials? Any problems? Are they useful? 



Are you grouping more this y'^ar than last year? How are you effecting it? 
Would you do moj;e if you coul<a? If yes, what prevents you? 



♦ 
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DMP INTERVIEW—SAMPLE 2 
Name School J_ 



Position Interviewer 



Note to Interviewer; If the person Sconpled was also drawn in Sample 1, some^of 

- these questions will be repetitive* Just explain that 
they should give their current response to the question; 
this is how "sampling" works. 

PRINCIPAL (omit this section at Morgan Martin ahd P^rkman) 

1. What costs other than Rand printed materials were, necessary for you to continue 
DMP at K-1? (e*g* , more manipulatives, replacement of broken items, e€c,) 



2. What is the reaction to DMP of 
teachers? 

1 

pupils? 



parents? 



o ■ Fit 
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Any special effort made to inform parents about DMP? 

V . 



3. Is there any difference ^his year in the attitudes of teachers (pupils, 
t>arents) who were involved in DMP last year? 




4. What effect, if any, on the teaching approach of- teachers does DMP have? 
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2nd GRADE TEACHER 



1. Have you had problems using DMP this year? (problems with teacher or student 
materials, directions for teachers, recommended teaching processes) 



12. Did you teach DMP last year? Yes No 



Have you noticed any advantage to using DMP this year? (enthusiasm of 
•students, more learning, transfer principles to other subjects) 



Er|c ' '^3 



Which, if any, of the assessment materials do you use? (Placement Inventory, 
Observation Schedule, Topic Inventory, Topic Checklists, Individual Progress 
Sheets, Group Record Cards) If so, are they useful? Any ^problems? (time, ' 
ease of administration) If not, how do you evaluate the children in DMP? 



If the Topic Checklists or Group Record Cards are used, make a judgment of a 
proportion of your DMP students attaining mastery/progressing/non-mastery 
for each objective instructed this year (e.g., 20% ft, 80% P, 0% N) . 



fi4 
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'1 > 

% 

If you gave a t>laceinerft Inventory, what use did you make of the results 
(e*g*, used them as general background information but not to group; used 
them to form groups; did not gtoup but used information to omit or include 
certain objectives) ? 



A 



Do you have groups of students working on different topics at the same tfme 

Do you have groups of students working at different activities. 

within a topic at the same time? ' How do you form instructional" 

groups? What size arc the gVoups typicJflly? 



Is any use made of *the Instructional Programming ideas in your Guide? 
Specify: 



Would you do'' more group of different- varieties if you could? ^ 
If yes, what prevents you? ^ ' • 



V 
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7. Did you confer or ^eceiv6 infornjation from last year's teacher regarding 
the student's progress in C^^? ' If so what and how did you use it? (Look 
for starting up where they left off, or using a Placement Inventory, or 
teacher judgment.) 



/ 



TEACHER WHO USED DMP LAST YEAR 



1. What is your reaction to the revised DMP^jnaterials? (improved, about the 
same, worse; effect on teaching) Specify what you like and dislike — format 
workbooks, etc. / 
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Is your use of DMP any different this year, e.g., teaching style, questioning 
techniques; use of small groups? (Try to sift out one yearns experience if 
possible; i,e,, compare fall 1972 to last winter,) 



Do you feel your attitude toward DMP is any different this .year? 



How? 
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4*» Are you using the assessment materials? (See question #3 for 2nd Grade 
•Teacher.) Any problems? Are they useful?* 



r 



5. Afe you grouping more this year than last year? How are you effecting it? 
Would you do more if you could? If yes, what prevents you? 



6* What procedures hOvVe you developed for reporting pupil progress to parent's 
"(e.g*, convert M^Pf N to grades; show parents Topic Checklists; write up a 
narrative; use a standard report card checklist)? 



REMINDER: COLLECT COMMENT CARDS 1 

ERJC ^ - ^ 
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Formative Conmeats 
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As usual/ the ' comments documented here are reported at "fac^ value." 
A3,though a, few comments regarding assessment materials are included, these 
were generally sent directly on to the project staff along with topic inventory 
response data. Unless specified, the comments about a particular activity were 
offered by a single teacher; however, most comments of a general nature (e.g., 
about an entire level or topic) ,came from unit leaders in multiunit schools. 
» and thus can be considered representative of the viewpoint of several teachers. 



Level _3 , ^ 

General Comments . v 

Need more work on zeros in sentence-writing topics. 
Recommend more use of counters with overhead projector. 
4 Some confusion between empty^^set and -shaded set (format). 

Develop ajn extra booklet (e.g.. Part IV) of independent activities, 

mixing review, "dr\ll," etc. 
•Offer more variation in activity choices with regard to achievement, 
so pace for individual stiJi^ents can be more varied. 



Topics 3.1 and 3.3 — Combine them into one topic. ■ ' . >^ 

Students have a great tendency to equalize rather than join (separate). 
For example, in a problem in which a child has one balloon, then gives. 
It one ayffay, he writes 1 + 0 = 1- 

Why not have both vertical and horizontal formats? (Qr, explain some- 
where that vertical format is "saved" in DMP for coi^iputation. ) 

Topic 3.3 — Moves from objects to symtjols too fast. Number stories too 
complex. Students still confusing addition and subtraction. 

Students should draw more pictures. 
Too nuny different worksheet formats. 
Topic too sophisticated, complex. 

Not enough isolated, concentrated work on each sentence type. 
Make objectives preparatory. 

Activity 3.3.9a-c — Story problems 

Wording ci^uses cpnfusion as do questiort$ asking for irrelevant answers. 
Since this is eariy in equalization work, have the irrelevant information 
questions in the latter problems. Would prefer those in workbook form 
so students can read along. 

V Activity 3.3.10 — Too long and complicated. 

Activity 3.4*5, 3.417 — Suggest answers for the teachers for the cards. 

Teachers feel they have to figure out the station answers ahead of time. 
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Topic 3.§ 

Activity 3.5.8, worJcbook pages 107-8 

Not nearly enough worksheets in this format. Not enough work on the 
un3cnown'in the first position — very troublesome. ' 

Topic Inventory 5 

The problems annotated for activity 3.5^8 cause difficulties in the 
Topic Inventory, since it uses this^ , format. 

Topic Inve^itories 3.3, 3.5, 3.7 

Don't correspond completely to, topic formats. 

Activity 3.7.2 

Vertical format should be introduced earlier. Would it be OK to tell 

students 4 + 4 = +^? ' • ' ^' 

4 

Activity 3.7.5, workbook page 138 - • 

Warn teachers that no half-circle piece is available. " 
Bevel _2 

GenQj^al Comment — The 1972-73 revision has less work on equalizing — need more. 
Topic 2.6 

, ^ .".^ 

Too many activities required. Make activities more /'to the point." 

Level i, J . • 

a.6.2B . • *' ' 

An inner city teacher reported the "pirate" context was beyond her students' 
experience. Story 1.6.2 was a great success on the other hand. 

1.8.8, workbook pages 67-68, 69 — See attached. These comments were offered 
independently by teachers at two schools. 

^.P. 67-68 are confusing 'because there are so many different things on 
the page. • ^ ^ 

Topic Inventory 1^8 ^ ^ ' * 



this. * * 
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Complaint that one "has to be a magician, using sle'ight of hand" to give 
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I.IO.S, workbook pages 93, 94 — See attached. 

"Generally poorly designee?,. ' , 
Other- — Record forms 



c 



Suggest that they b< orderable separately or be packaged originally in 
larger quantities since they are consumable. 

Should indicate on order blank or in publicaiioiis the quantity; of records 
*(and number of students accommodated) that will be sent with printed 
materials. ' 



Levels 1-2 



Individual assessments still too time-consuming. 



i 
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VICTORY SCHOOL 
MAV 1^73 

\. THE DISTANCE BETuJEEN CAQMOT AHl> 
1.10.5 CABBITS IS TOO SIMIUAR INCH). 

2. 06ING 5TRlh\Gj A5 Pi&^CTEP IS iMPOSglg Lg, 
IM MOST CASES. VJE USED STIC*:y" 
. TAPE 1^ AND E\)EN THAT uJAS DlFFlCUt-T. 

\ 3. THf VCINPEROACTEKJ CH lUD^S 

-^COOCPlHATIONl IS MOT fiOFFlCieNTLV 
PEVEUOPEP TD HAiOPLE THIS. THE pICrTUIZES- 
AttE TOO SMAU- , 

^. THE pieEcTioNs Aiae irWAccueATE AS you 

CANNOT PASTE THE STEJNC? ON TH& OTHEl^ 
SIPE. 




VUS GOVERNMENT PRINTING CCFFICE 1»75-«5(M«4 1152 

DMP MATHEMATICS XEVEL ONE 93 
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